{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "79085caa",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/peremartra/Large-Language-Model-Notebooks-Course/blob/main/3-LangChain/3_1_RAG_langchain.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6JEdfdtDgfaz",
   "metadata": {
    "id": "6JEdfdtDgfaz"
   },
   "source": [
    "<div>\n",
    "<h1>Large Language Models Projects</h1>\n",
    "    <h3>Apply and Implement Strategies for Large Language Models</h3>\n",
    "    <h2>3.1-Use the Data from your DataFrames with LLMs.</h2>\n",
    "    \n",
    "</div>\n",
    "\n",
    "by [Pere Martra](https://www.linkedin.com/in/pere-martra/)\n",
    "\n",
    "_______________\n",
    "Models: dolly-v2-3b / flan-t5-large\n",
    "\n",
    "Colab Environmet: CPU - High RAM\n",
    "\n",
    "Keys:\n",
    "* RAG\n",
    "* LangChain\n",
    "* Embeddings\n",
    "* LCEL\n",
    "\n",
    "Article related: [Query Your DataFrames with Powerful Large Language Models using LangChain.](https://pub.towardsai.net/query-your-dataframes-with-powerful-large-language-models-using-langchain-abe25782def5)\n",
    "_________________\n",
    "\n",
    "If you are executing this notebook on Colab you will need a High RAM capacity environment.\n",
    "\n",
    "If you don't have a Colab Pro acoount you can execute this notebook on kaggle, since you will get more memory from the free tier."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b863e355",
   "metadata": {
    "id": "b863e355",
    "papermill": {
     "duration": 0.01084,
     "end_time": "2023-11-07T23:38:48.872411",
     "exception": false,
     "start_time": "2023-11-07T23:38:48.861571",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# TUTORIAL HOW TO USE LANGCHAIN AND VECTOR DATABASES TO USE OUR OWN DOCUMENTS WITH HUGGING FACES\n",
    "\n",
    "In a [previous notebook](https://www.kaggle.com/code/peremartramanonellas/use-a-vectorial-db-to-optimize-prompts-for-llms), we saw how to use a vectorial database to create an enriched prompt for Hugging Face language models. In this one, we incorporate LangChain so that we can make queries to the language model that take into account our information.\n",
    "\n",
    "The information could be our own documents, or whatever was contained in a business knowledge database.\n",
    "\n",
    "I have prepared the notebook so that it can work with  different Kaggle datasets, so that it is easy to carry out different tests with different Datasets.\n",
    "\n",
    "The data has been loaded from a Pandas DataFrame, using the **DataFrameLoader** function from the **document_loaders** library of **LangChain**.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bac2f0d",
   "metadata": {
    "id": "5bac2f0d",
    "papermill": {
     "duration": 0.010703,
     "end_time": "2023-11-07T23:38:48.894198",
     "exception": false,
     "start_time": "2023-11-07T23:38:48.883495",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Install and load the libraries.\n",
    "To start we need to install the necesary Python packages.\n",
    "* **[langchain](https://python.langchain.com/docs/get_started/introduction.html)**. The revolutionary framework to build apps using large language models.\n",
    "* **[sentence_transformers](https://www.sbert.net/)**. necesary to create the embeddings we are going to store in the vector database.  \n",
    "* **[chromadb](https://www.trychroma.com/)**. This is our vector Database. ChromaDB is easy to use and open source, maybe the most used Vector Database used to store embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ecaf9c5",
   "metadata": {
    "_kg_hide-output": true,
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "execution": {
     "iopub.execute_input": "2023-11-07T23:38:48.918551Z",
     "iopub.status.busy": "2023-11-07T23:38:48.917508Z",
     "iopub.status.idle": "2023-11-07T23:40:10.466442Z",
     "shell.execute_reply": "2023-11-07T23:40:10.465295Z"
    },
    "id": "1ecaf9c5",
    "outputId": "502a6e6e-4523-4d89-9e73-46c37693417c",
    "papermill": {
     "duration": 81.563917,
     "end_time": "2023-11-07T23:40:10.469187",
     "exception": false,
     "start_time": "2023-11-07T23:38:48.90527",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip install -q chromadb==0.4.22\n",
    "!pip install -q langchain==0.1.4\n",
    "!pip install -q sentence_transformers==2.3.0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f407d757",
   "metadata": {
    "id": "f407d757",
    "papermill": {
     "duration": 0.022555,
     "end_time": "2023-11-07T23:40:10.609309",
     "exception": false,
     "start_time": "2023-11-07T23:40:10.586754",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Load the Dataset\n",
    "As you can see the notebook is ready to work with different Datasets. Just uncomment the lines of the Dataset you want to use.\n",
    "\n",
    "As we are working in a free and limited space, and we can use just some limited gb of memory I limited the number of news to use with the variable MAX_NEWS.\n",
    "\n",
    "The name of the field containing the text of the new is stored in the variable *DOCUMENT* and the metadata in *TOPIC*\n",
    "\n",
    "I used the kotartemiy/topic-labeled-news-dataset https://www.kaggle.com/datasets/kotartemiy/topic-labeled-news-dataset\n",
    "\n",
    "Artem Burgara. (2020). R vs. Python: Topic Labeled News Dataset, . Retrieved December 2023, from https://www.kaggle.com/discussions/general/46091.\n",
    "\n",
    "But you can ose other datasets, I encourage you to try at least one of these:\n",
    "\n",
    "https://www.kaggle.com/datasets/gpreda/bbc-news\n",
    "\n",
    "https://www.kaggle.com/datasets/deepanshudalal09/mit-ai-news-published-till-2023\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "OakH4pOdQrlo",
   "metadata": {
    "id": "OakH4pOdQrlo"
   },
   "source": [
    "### Conecting to Kaggle."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "QV09oMxbZayb",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "QV09oMxbZayb",
    "outputId": "47c9f426-a772-45b7-ce8f-f33726201aef"
   },
   "outputs": [],
   "source": [
    "from google.colab import drive\n",
    "drive.mount('/content/drive')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7nOpQ8StZk9W",
   "metadata": {
    "id": "7nOpQ8StZk9W"
   },
   "outputs": [],
   "source": [
    "!pip install -q kaggle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "-FH7rMgFZ0rm",
   "metadata": {
    "id": "-FH7rMgFZ0rm"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "#This directory should contain you kaggle.json file with your key\n",
    "os.environ['KAGGLE_CONFIG_DIR'] = '/content/drive/MyDrive/kaggle'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "F7A8FzbcZ3t6",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "F7A8FzbcZ3t6",
    "outputId": "023e6a75-7fcc-4694-90f5-02c9e367174c"
   },
   "outputs": [],
   "source": [
    "!kaggle datasets download -d kotartemiy/topic-labeled-news-dataset\n",
    "#!kaggle datasets download -d gpreda/bbc-news"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "09ea9d7c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:10.558175Z",
     "iopub.status.busy": "2023-11-07T23:40:10.55769Z",
     "iopub.status.idle": "2023-11-07T23:40:10.562952Z",
     "shell.execute_reply": "2023-11-07T23:40:10.561864Z"
    },
    "id": "09ea9d7c",
    "papermill": {
     "duration": 0.030688,
     "end_time": "2023-11-07T23:40:10.565135",
     "exception": false,
     "start_time": "2023-11-07T23:40:10.534447",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "OPZ3TbXlaWXV",
   "metadata": {
    "id": "OPZ3TbXlaWXV"
   },
   "outputs": [],
   "source": [
    "import zipfile\n",
    "\n",
    "# Define the path to your zip file\n",
    "file_path = '/content/topic-labeled-news-dataset.zip'\n",
    "#file_path = '/content/bbc-news.zip'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "seBuoNVuaaTw",
   "metadata": {
    "id": "seBuoNVuaaTw"
   },
   "outputs": [],
   "source": [
    "with zipfile.ZipFile(file_path, 'r') as zip_ref:\n",
    "    zip_ref.extractall('/content/drive/MyDrive/kaggle')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34403557",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:10.653256Z",
     "iopub.status.busy": "2023-11-07T23:40:10.652861Z",
     "iopub.status.idle": "2023-11-07T23:40:11.635339Z",
     "shell.execute_reply": "2023-11-07T23:40:11.634406Z"
    },
    "id": "34403557",
    "papermill": {
     "duration": 1.007443,
     "end_time": "2023-11-07T23:40:11.637872",
     "exception": false,
     "start_time": "2023-11-07T23:40:10.630429",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "news = pd.read_csv('/content/drive/MyDrive/kaggle/labelled_newscatcher_dataset.csv', sep=';')\n",
    "MAX_NEWS = 1000\n",
    "DOCUMENT=\"title\"\n",
    "TOPIC=\"topic\"\n",
    "\n",
    "#news = pd.read_csv('/content/drive/MyDrive/kaggle/bbc_news.csv')\n",
    "#MAX_NEWS = 500\n",
    "#DOCUMENT=\"description\"\n",
    "#TOPIC=\"title\"\n",
    "\n",
    "#Because it is just a course we select a small portion of News.\n",
    "subset_news = news.head(MAX_NEWS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87b2f5bd",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 112
    },
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:11.682843Z",
     "iopub.status.busy": "2023-11-07T23:40:11.682429Z",
     "iopub.status.idle": "2023-11-07T23:40:11.703593Z",
     "shell.execute_reply": "2023-11-07T23:40:11.702566Z"
    },
    "id": "87b2f5bd",
    "outputId": "86a30252-add3-4455-9d81-23b1ecae7598",
    "papermill": {
     "duration": 0.046286,
     "end_time": "2023-11-07T23:40:11.705855",
     "exception": false,
     "start_time": "2023-11-07T23:40:11.659569",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "news.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25ef4e81",
   "metadata": {
    "id": "25ef4e81",
    "papermill": {
     "duration": 0.021487,
     "end_time": "2023-11-07T23:40:11.748796",
     "exception": false,
     "start_time": "2023-11-07T23:40:11.727309",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## CREATE THE DOCUMENT FROM THE DATAFRAME\n",
    "We are going to load the data from a pandas DataFrame. However, LangChain, through the **document_loader** library, supports multiple data sources, such as Word documents, Excel files, plain text, SQL, and more.\n",
    "\n",
    "We also imported the Chroma library, which is used to save the embeddings in the ChromaDB database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c6f1238",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:11.793285Z",
     "iopub.status.busy": "2023-11-07T23:40:11.792911Z",
     "iopub.status.idle": "2023-11-07T23:40:12.353535Z",
     "shell.execute_reply": "2023-11-07T23:40:12.352343Z"
    },
    "id": "4c6f1238",
    "papermill": {
     "duration": 0.585876,
     "end_time": "2023-11-07T23:40:12.35627",
     "exception": false,
     "start_time": "2023-11-07T23:40:11.770394",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.document_loaders import DataFrameLoader\n",
    "from langchain.vectorstores import Chroma\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb88b124",
   "metadata": {
    "id": "eb88b124",
    "papermill": {
     "duration": 0.021508,
     "end_time": "2023-11-07T23:40:12.399239",
     "exception": false,
     "start_time": "2023-11-07T23:40:12.377731",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "First, we create the loader, indicating the data source and the name of the column in the DataFrame where we store what we could consider as the document, that is, the information we want to pass to the model so that it takes it into account in its responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "78614daa",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:12.444219Z",
     "iopub.status.busy": "2023-11-07T23:40:12.443484Z",
     "iopub.status.idle": "2023-11-07T23:40:12.449168Z",
     "shell.execute_reply": "2023-11-07T23:40:12.447976Z"
    },
    "id": "78614daa",
    "papermill": {
     "duration": 0.030632,
     "end_time": "2023-11-07T23:40:12.451276",
     "exception": false,
     "start_time": "2023-11-07T23:40:12.420644",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_loader = DataFrameLoader(subset_news, page_content_column=DOCUMENT)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e777e3ec",
   "metadata": {
    "id": "e777e3ec",
    "papermill": {
     "duration": 0.02084,
     "end_time": "2023-11-07T23:40:12.4936",
     "exception": false,
     "start_time": "2023-11-07T23:40:12.47276",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Then, we use the loader to load the document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d0b81fd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:12.537857Z",
     "iopub.status.busy": "2023-11-07T23:40:12.537404Z",
     "iopub.status.idle": "2023-11-07T23:40:12.652204Z",
     "shell.execute_reply": "2023-11-07T23:40:12.650913Z"
    },
    "id": "1d0b81fd",
    "papermill": {
     "duration": 0.140406,
     "end_time": "2023-11-07T23:40:12.65498",
     "exception": false,
     "start_time": "2023-11-07T23:40:12.514574",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_document = df_loader.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "948950f7",
   "metadata": {
    "_kg_hide-output": true,
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 121
    },
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:12.700437Z",
     "iopub.status.busy": "2023-11-07T23:40:12.699647Z",
     "iopub.status.idle": "2023-11-07T23:40:12.759691Z",
     "shell.execute_reply": "2023-11-07T23:40:12.758871Z"
    },
    "id": "948950f7",
    "outputId": "9b1ccdff-c919-4cc7-a9f3-205f2228d494",
    "papermill": {
     "duration": 0.10378,
     "end_time": "2023-11-07T23:40:12.780315",
     "exception": false,
     "start_time": "2023-11-07T23:40:12.676535",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "display(df_document[:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82937c96",
   "metadata": {
    "id": "82937c96",
    "papermill": {
     "duration": 0.026601,
     "end_time": "2023-11-07T23:40:12.833936",
     "exception": false,
     "start_time": "2023-11-07T23:40:12.807335",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Creating the embeddings\n",
    "First, we import a couple of libraries.\n",
    "* CharacterTextSplitter: we will use it to group the information contained in different blocks.\n",
    "* HuggingFaceEmbeddings: it will create the embeddings in the format that we will store in the database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6963280e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:12.889831Z",
     "iopub.status.busy": "2023-11-07T23:40:12.888855Z",
     "iopub.status.idle": "2023-11-07T23:40:12.893122Z",
     "shell.execute_reply": "2023-11-07T23:40:12.892384Z"
    },
    "id": "6963280e",
    "papermill": {
     "duration": 0.034128,
     "end_time": "2023-11-07T23:40:12.895046",
     "exception": false,
     "start_time": "2023-11-07T23:40:12.860918",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.text_splitter import CharacterTextSplitter\n",
    "#from langchain.embeddings import HuggingFaceEmbeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b102d83f",
   "metadata": {
    "id": "b102d83f",
    "papermill": {
     "duration": 0.026522,
     "end_time": "2023-11-07T23:40:12.948396",
     "exception": false,
     "start_time": "2023-11-07T23:40:12.921874",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "As I said above we split the data into manageable chunks to store as vectors using **CharacterTextSplitter**. There isn't an exact way to do this, more chunks means more detailed context, but will increase the size of our vectorstore.\n",
    "\n",
    "There are no magic numbers to inform. It is important to consider that the larger the chunk size, the more context the model will have, but the size of our vector store will also increase."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "316f5b14",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:13.004502Z",
     "iopub.status.busy": "2023-11-07T23:40:13.003754Z",
     "iopub.status.idle": "2023-11-07T23:40:13.04734Z",
     "shell.execute_reply": "2023-11-07T23:40:13.046217Z"
    },
    "id": "316f5b14",
    "papermill": {
     "duration": 0.07506,
     "end_time": "2023-11-07T23:40:13.050342",
     "exception": false,
     "start_time": "2023-11-07T23:40:12.975282",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "text_splitter = CharacterTextSplitter(chunk_size=250, chunk_overlap=10)\n",
    "texts = text_splitter.split_documents(df_document)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7c982e5e",
   "metadata": {
    "_kg_hide-output": true,
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 121
    },
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:13.106064Z",
     "iopub.status.busy": "2023-11-07T23:40:13.105457Z",
     "iopub.status.idle": "2023-11-07T23:40:13.160232Z",
     "shell.execute_reply": "2023-11-07T23:40:13.158852Z"
    },
    "id": "7c982e5e",
    "outputId": "ac42cf20-65ee-4e9d-ab70-0b855f6f8781",
    "papermill": {
     "duration": 0.103411,
     "end_time": "2023-11-07T23:40:13.18066",
     "exception": false,
     "start_time": "2023-11-07T23:40:13.077249",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "display(texts[:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "779e3387",
   "metadata": {
    "id": "779e3387",
    "papermill": {
     "duration": 0.03243,
     "end_time": "2023-11-07T23:40:13.245472",
     "exception": false,
     "start_time": "2023-11-07T23:40:13.213042",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "We load the library to create the pre trained model from HuggingFace to create the embeddings from sentences.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7fa973ac",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:13.31152Z",
     "iopub.status.busy": "2023-11-07T23:40:13.310846Z",
     "iopub.status.idle": "2023-11-07T23:40:31.996557Z",
     "shell.execute_reply": "2023-11-07T23:40:31.995022Z"
    },
    "id": "7fa973ac",
    "outputId": "0898af45-3e70-47cb-cd07-859afbe1d043",
    "papermill": {
     "duration": 18.721786,
     "end_time": "2023-11-07T23:40:31.999291",
     "exception": false,
     "start_time": "2023-11-07T23:40:13.277505",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.embeddings.sentence_transformer import SentenceTransformerEmbeddings\n",
    "embedding_function = SentenceTransformerEmbeddings(model_name=\"all-MiniLM-L6-v2\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea58c45e",
   "metadata": {
    "id": "ea58c45e",
    "papermill": {
     "duration": 0.034795,
     "end_time": "2023-11-07T23:40:32.069162",
     "exception": false,
     "start_time": "2023-11-07T23:40:32.034367",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Creating the Index With Chroma\n",
    "Here we are creating the index of embeddings. Using the document, and the embedding function created above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eadc6d53",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:32.142168Z",
     "iopub.status.busy": "2023-11-07T23:40:32.140958Z",
     "iopub.status.idle": "2023-11-07T23:40:40.614344Z",
     "shell.execute_reply": "2023-11-07T23:40:40.613017Z"
    },
    "id": "eadc6d53",
    "papermill": {
     "duration": 8.512528,
     "end_time": "2023-11-07T23:40:40.616895",
     "exception": false,
     "start_time": "2023-11-07T23:40:32.104367",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "directory_cdb = '/content/drive/MyDrive/chromadb'\n",
    "chroma_db = Chroma.from_documents(\n",
    "    texts, embedding_function, persist_directory=directory_cdb\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "708820c5",
   "metadata": {
    "id": "708820c5",
    "papermill": {
     "duration": 0.034723,
     "end_time": "2023-11-07T23:40:40.687547",
     "exception": false,
     "start_time": "2023-11-07T23:40:40.652824",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## LANGCHAIN\n",
    "\n",
    "Finally, the time has come to create our chain with LangChain. It will be straightforward. All we do is give it a retriever and a model to call with the result obtained from the retriever.\n",
    "\n",
    "Now we are going to import RetrievalQA and HuggingFacePipeline classes from langchain module.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae871f01",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:40.762362Z",
     "iopub.status.busy": "2023-11-07T23:40:40.761944Z",
     "iopub.status.idle": "2023-11-07T23:40:42.256159Z",
     "shell.execute_reply": "2023-11-07T23:40:42.254902Z"
    },
    "id": "ae871f01",
    "papermill": {
     "duration": 1.534176,
     "end_time": "2023-11-07T23:40:42.258996",
     "exception": false,
     "start_time": "2023-11-07T23:40:40.72482",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.chains import RetrievalQA\n",
    "from langchain.llms import HuggingFacePipeline\n",
    "#from transformers import pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da545337",
   "metadata": {
    "id": "da545337",
    "papermill": {
     "duration": 0.035218,
     "end_time": "2023-11-07T23:40:42.330255",
     "exception": false,
     "start_time": "2023-11-07T23:40:42.295037",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Now we create the retriever object, the responsible to return the data contained in the ChromaDB Database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a8478f13",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:42.40325Z",
     "iopub.status.busy": "2023-11-07T23:40:42.402545Z",
     "iopub.status.idle": "2023-11-07T23:40:42.408031Z",
     "shell.execute_reply": "2023-11-07T23:40:42.406852Z"
    },
    "id": "a8478f13",
    "papermill": {
     "duration": 0.04437,
     "end_time": "2023-11-07T23:40:42.410217",
     "exception": false,
     "start_time": "2023-11-07T23:40:42.365847",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "retriever = chroma_db.as_retriever()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eadae843",
   "metadata": {
    "id": "eadae843",
    "papermill": {
     "duration": 0.03539,
     "end_time": "2023-11-07T23:40:42.481519",
     "exception": false,
     "start_time": "2023-11-07T23:40:42.446129",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "I tested the notebook with two models from Fugging Face.\n",
    "\n",
    "The first one is [dolly-v2-3b](https://huggingface.co/databricks/dolly-v2-3b), the smallest Dolly model. It have 3billion paramaters, more than enough for our sample, and works much better than GPT2. It's a text generation model, and therefore generates slightly more imaginative responses.\n",
    "\n",
    "The second one is a t5 model. This is a text2text-generation. so it will produce more concise and succinct responses.\n",
    "\n",
    "Just be sure the test both, and if you want select other models from Hugging Face."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "94fe7021",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:42.554566Z",
     "iopub.status.busy": "2023-11-07T23:40:42.554143Z",
     "iopub.status.idle": "2023-11-07T23:40:42.559719Z",
     "shell.execute_reply": "2023-11-07T23:40:42.558545Z"
    },
    "id": "94fe7021",
    "papermill": {
     "duration": 0.044891,
     "end_time": "2023-11-07T23:40:42.561978",
     "exception": false,
     "start_time": "2023-11-07T23:40:42.517087",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "model_id = \"databricks/dolly-v2-3b\" #a good textgeneration model for testing\n",
    "task=\"text-generation\"\n",
    "\n",
    "#model_id = \"google/flan-t5-large\" #Nice text2text model\n",
    "#task=\"text2text-generation\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bef59f8",
   "metadata": {
    "id": "4bef59f8",
    "papermill": {
     "duration": 0.03455,
     "end_time": "2023-11-07T23:40:42.631714",
     "exception": false,
     "start_time": "2023-11-07T23:40:42.597164",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "We use HuggingFacePipeline class to create a pipeline for a specific Hugging Face language model. Let's break down the code:\n",
    "\n",
    "* **model_id**: This is the ID of the Hugging Face language model you want to use. It typically consists of the model name and version.\n",
    "* **task**: This parameter specifies the task that you want to perform using the language model. It could be \"text-generation\", \"text2text-generation\", \"question-answering\", or other tasks supported by the model.\n",
    "* **model_kwargs**: Allows you to provide additional arguments specific to the chosen model. In this case, it sets \"temperature\" to 0 (indicating deterministic output) and \"max_length\" to 256, which limits the maximum length of generated text to 256 tokens.\n",
    "* **pipeline_kwargs**: Allows you to provide extra information related to the pipeline.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0bdfec13",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "execution": {
     "iopub.execute_input": "2023-11-07T23:40:42.704032Z",
     "iopub.status.busy": "2023-11-07T23:40:42.703576Z",
     "iopub.status.idle": "2023-11-07T23:42:17.458434Z",
     "shell.execute_reply": "2023-11-07T23:42:17.456989Z"
    },
    "id": "0bdfec13",
    "outputId": "588d7a01-f5c9-49b5-f08e-95d493437c24",
    "papermill": {
     "duration": 94.79476,
     "end_time": "2023-11-07T23:42:17.461674",
     "exception": false,
     "start_time": "2023-11-07T23:40:42.666914",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "#If you want to test the  T5 model remove the return_full_text parameter in pipeline_kwargs,\n",
    "#also I recommend to add model_kwargs and change the temperature.\n",
    "#    model_kwargs={\n",
    "#        \"temperature\": 0,\n",
    "#        \"max_length\": 256\n",
    "#    },\n",
    "hf_llm = HuggingFacePipeline.from_model_id(\n",
    "    model_id=model_id,\n",
    "    task=task,\n",
    "    pipeline_kwargs={\n",
    "        \"max_new_tokens\": 256,\n",
    "        \"repetition_penalty\":1.1,\n",
    "        \"return_full_text\":True\n",
    "    },\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5cc16f20",
   "metadata": {
    "id": "5cc16f20",
    "papermill": {
     "duration": 0.03792,
     "end_time": "2023-11-07T23:42:17.537572",
     "exception": false,
     "start_time": "2023-11-07T23:42:17.499652",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "We are setting up the ***document_qa***, a **RetrievalQA** object, that we are going to use to run the questions.\n",
    "\n",
    "The ***stuff*** type is the simplest type of chain that we can have. I get the documents from the retiever and use the language model to obtain responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa608d67",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T23:42:17.61509Z",
     "iopub.status.busy": "2023-11-07T23:42:17.61441Z",
     "iopub.status.idle": "2023-11-07T23:42:17.621618Z",
     "shell.execute_reply": "2023-11-07T23:42:17.620446Z"
    },
    "id": "aa608d67",
    "papermill": {
     "duration": 0.049131,
     "end_time": "2023-11-07T23:42:17.623996",
     "exception": false,
     "start_time": "2023-11-07T23:42:17.574865",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "document_qa = RetrievalQA.from_chain_type(\n",
    "    llm=hf_llm, retriever=retriever, chain_type='stuff'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c83a8eca",
   "metadata": {
    "id": "c83a8eca",
    "papermill": {
     "duration": 0.0364,
     "end_time": "2023-11-07T23:42:17.696923",
     "exception": false,
     "start_time": "2023-11-07T23:42:17.660523",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Time to call the chain and obtain the responses!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f25c34f0",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "execution": {
     "iopub.execute_input": "2023-11-07T23:42:17.772278Z",
     "iopub.status.busy": "2023-11-07T23:42:17.771797Z",
     "iopub.status.idle": "2023-11-07T23:43:36.092127Z",
     "shell.execute_reply": "2023-11-07T23:43:36.090889Z"
    },
    "id": "f25c34f0",
    "outputId": "91126027-bb57-4be6-ba52-b14be2278cce",
    "papermill": {
     "duration": 78.397935,
     "end_time": "2023-11-07T23:43:36.131687",
     "exception": false,
     "start_time": "2023-11-07T23:42:17.733752",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "#Sample question for newscatcher dataset.\n",
    "response = document_qa.invoke(\"Can I buy a Toshiba laptop?\")\n",
    "\n",
    "#Sample question for BBC Dataset.\n",
    "#response = document_qa.run(\"Who is going to meet boris johnson?\")\n",
    "display(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0nVxkDzLSMgr",
   "metadata": {
    "id": "0nVxkDzLSMgr"
   },
   "source": [
    "### Using the new LCEL Architecture from LangChain.\n",
    "LangChain recommends using LCEL (LangChain Expression Language) over Chains. I'm using both methods in the notebook, but please note that LCEL is the recommended approach."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ZtlFB5UhSMNl",
   "metadata": {
    "id": "ZtlFB5UhSMNl"
   },
   "outputs": [],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_core.runnables import RunnablePassthrough\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "\n",
    "template = \"\"\"Answer the question based on the following context:\n",
    "{context}\n",
    "\n",
    "Question: {question}\n",
    "\"\"\"\n",
    "prompt = ChatPromptTemplate.from_template(template)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "HwaLFCjxSpHF",
   "metadata": {
    "id": "HwaLFCjxSpHF"
   },
   "outputs": [],
   "source": [
    "chain = (\n",
    "    {\"context\": retriever, \"question\": RunnablePassthrough()}\n",
    "    | prompt\n",
    "    | hf_llm\n",
    "    | StrOutputParser()\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1RpaUJjhSp9y",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "id": "1RpaUJjhSp9y",
    "outputId": "df051d4f-3c18-4a55-8f7e-5e3476679284"
   },
   "outputs": [],
   "source": [
    "chain.invoke(\"Can I buy a Toshiba laptop?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mwSm-uLnaY3k",
   "metadata": {
    "id": "mwSm-uLnaY3k"
   },
   "source": [
    "## Checking the cosine distance between sentences.\n",
    "\n",
    "Bonus section: As you may know, vector databases find the most relevant information to a user's question by measuring the distance between embeddings.\n",
    "\n",
    "Here's a brief example to illustrate how this works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "hja5kK6daY3l",
   "metadata": {
    "id": "hja5kK6daY3l"
   },
   "outputs": [],
   "source": [
    "embedding_s1 = embedding_function.embed_query(\n",
    "    \"I would like to eat more vegetables and exercise every day\")\n",
    "\n",
    "embedding_s2 = embedding_function.embed_query(\n",
    "    \"I will try to maintain a healthier lifestyle.\")\n",
    "\n",
    "embedding_s3 = embedding_function.embed_query(\n",
    "    \"I prefer to play football\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0GfRJLVcaY3l",
   "metadata": {
    "id": "0GfRJLVcaY3l"
   },
   "source": [
    "All the embeddings have the same lenght despite the sentence length."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1FjwxUxTaY3m",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "1FjwxUxTaY3m",
    "outputId": "10b67ab5-4cd0-4cc0-c1f9-6abc98cd1cf3"
   },
   "outputs": [],
   "source": [
    "print(f\"\"\" embedding_s1 = {len(embedding_s1)}\n",
    " embedding_s2 =  {len(embedding_s2)}\n",
    " embedding_s3 =  {len(embedding_s3)}\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "Pi6MkKKYaY3m",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Pi6MkKKYaY3m",
    "outputId": "cb930085-c4fd-4bd5-ab15-c829be982c49"
   },
   "outputs": [],
   "source": [
    "# 5 first positions of embedding_s1.\n",
    "print(embedding_s1[:5])\n",
    "print(embedding_s2[:5])\n",
    "print(embedding_s3[:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "MxoqDpLIaY3m",
   "metadata": {
    "id": "MxoqDpLIaY3m"
   },
   "source": [
    "Importing SKLEARN to measure the cosine distances"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fmS-ZYiHi7f-",
   "metadata": {
    "id": "fmS-ZYiHi7f-"
   },
   "outputs": [],
   "source": [
    "!pip install -q scikit-learn==1.2.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ntmp1iflaY3m",
   "metadata": {
    "id": "ntmp1iflaY3m"
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics.pairwise import cosine_similarity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "zWS2m7J3aY3m",
   "metadata": {
    "id": "zWS2m7J3aY3m"
   },
   "outputs": [],
   "source": [
    "embedding_s1_2d = np.array(embedding_s1).reshape(1, -1)\n",
    "embedding_s2_2d = np.array(embedding_s2).reshape(1, -1)\n",
    "embedding_s3_2d = np.array(embedding_s3).reshape(1, -1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eEz0wqn_aY3m",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "eEz0wqn_aY3m",
    "outputId": "41b3933d-0713-4a06-8706-db4b1d1be0df"
   },
   "outputs": [],
   "source": [
    "print(embedding_s1_2d[0][:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "VCKP78yEaY3n",
   "metadata": {
    "id": "VCKP78yEaY3n"
   },
   "source": [
    "S1 sentence is more similar to S2 sentence than to S3 sentence. That is because the first sentences are talking about a healthier way of live."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "PcEhD2JGaY3n",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "PcEhD2JGaY3n",
    "outputId": "bca12b77-e26a-462d-b966-47924d98a8c7"
   },
   "outputs": [],
   "source": [
    "print(cosine_similarity(embedding_s1_2d, embedding_s2_2d))\n",
    "print(cosine_similarity(embedding_s1_2d, embedding_s3_2d))\n",
    "print(cosine_similarity(embedding_s2_2d, embedding_s3_2d))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mncNTu_oe2i6",
   "metadata": {
    "id": "mncNTu_oe2i6"
   },
   "source": [
    "## Print embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "Jjculuboe1tA",
   "metadata": {
    "id": "Jjculuboe1tA"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from sklearn.decomposition import PCA\n",
    "\n",
    "# Perform PCA for 2D visualization\n",
    "PCA_model = PCA(n_components = 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "HA_2tUU6hIoE",
   "metadata": {
    "id": "HA_2tUU6hIoE"
   },
   "outputs": [],
   "source": [
    "embeddings_sentences=[]\n",
    "embeddings_sentences.append(embedding_s1)\n",
    "embeddings_sentences.append(embedding_s2)\n",
    "embeddings_sentences.append(embedding_s3)\n",
    "PCA_model.fit(embeddings_sentences)\n",
    "embeddings_coord = PCA_model.transform(embeddings_sentences)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "rzKn5vd3iqQa",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "rzKn5vd3iqQa",
    "outputId": "99cd1b3c-ba1c-44e1-ec83-4e39002ddbb5"
   },
   "outputs": [],
   "source": [
    "print(embeddings_coord)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "L8Wbyjnyj3KU",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 449
    },
    "id": "L8Wbyjnyj3KU",
    "outputId": "7c109a88-b27f-4e54-9c2c-d9bf801655ff"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "x, y = zip(*embeddings_coord)\n",
    "\n",
    "# Name the points\n",
    "names = ['s1', 's2', 's3']\n",
    "# Colors\n",
    "colors = ['red', 'green', 'blue']\n",
    "\n",
    "for i, name in enumerate(names):\n",
    "    plt.scatter(x[i], y[i], marker='o', color=colors[i], label=name)\n",
    "\n",
    "plt.xlabel('X')\n",
    "plt.ylabel('Y')\n",
    "plt.legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5387b3e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-07-12T22:01:56.993351Z",
     "iopub.status.busy": "2023-07-12T22:01:56.992775Z",
     "iopub.status.idle": "2023-07-12T22:01:57.001309Z",
     "shell.execute_reply": "2023-07-12T22:01:56.999431Z",
     "shell.execute_reply.started": "2023-07-12T22:01:56.993305Z"
    },
    "id": "c5387b3e",
    "papermill": {
     "duration": 0.037288,
     "end_time": "2023-11-07T23:43:36.206248",
     "exception": false,
     "start_time": "2023-11-07T23:43:36.16896",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Conclusions.\n",
    "Not a long notebook, but with a lot of content.\n",
    "\n",
    "We have used a vectorial database to store the information from a Kaggle dataset. We have incorporated it into a LangChain chain through a retriever, and now we are able to make queries about the information contained in the dataset to a couple of Hugging Face Language Models.\n",
    "\n",
    "And Also you've been introduce to LCEL!!!\n",
    "\n",
    "Please don't stop here.\n",
    "\n",
    "* The notebook is prepared to use two Datasets. Do tests with it.An if you have time select other Datset from Kaggle.\n",
    "\n",
    "* Find another model on Hugging Face and compare it.\n",
    "\n",
    "* We loaded the data from a Dataframe, try to upload Data from a .txt file.\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "M3Dgo0ISVHhE",
   "metadata": {
    "id": "M3Dgo0ISVHhE"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "include_colab_link": true,
   "machine_shape": "hm",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 301.368093,
   "end_time": "2023-11-07T23:43:39.05917",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2023-11-07T23:38:37.691077",
   "version": "2.4.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
